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EXAMINER'S ANSWER 



This is in response to the appeal brief filed 20 September 2005 appealing from the 
Office action mailed 1 June 2005. 
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(1) Real Party in Interest 

A statement identifying by name the real party in interest is contained in the brief. 

(2) Related Appeals and Interferences 

The examiner is not aware of any related appeals, interferences, or judicial 
proceedings which will directly affect or be directly affected by or have a bearing on the 
Board's decision in the pending appeal. 

(3) Status of Claims 

The statement of the status of claims contained in the brief is correct. 

(4) Status of Amendments After Final 

The appellant's statement of the status of amendments after final rejection 
contained in the brief is correct. 

(5) Summary of Claimed Subject Matter 

The summary of claimed subject matter contained in the brief is correct. 

(6) Grounds of Rejection to be Reviewed on Appeal 

The appellant's statement of the grounds of rejection to be reviewed on appeal is 
substantially correct. The changes are as follows: 

As per part B., it is only claims 1 , 4 - 6, 8 - 10, 43, 44, 52 and 53 that have been 
rejected under 35 USC 102(e) as being anticipated by Jain et al., owing to the 
cancellation of claims 2, 3. 

(7) Claims Appendix 

The copy of the appealed claims contained in the Appendix to the brief is correct. 
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(8) Evidence Relied Upon 



6,567,980 



JAIN et al. 



5-2003 



2002/0,088,000 



MORRIS 



7-2002 



6,813,745 



DUNCOMBE 



11-2004 



(9) Grounds of Rejection 

The following ground(s) of rejection are applicable to the appealed claims: 
a. Claims 45 - 47 are rejected under 35 U.S.C. 101 because the claimed invention 
is directed to non-statutory subject matter. 

These claims are directed to a "computer-readable medium having computer 
executable instructions for employing a data packet" (claims 45, 47) or "memory" (claim 
46), with a further limitation upon the contents of the "packet" or "memory". 

However, this does not result in a statutory "machine" or "article of manufacture" 
claim, when the claim is considered in light of applicant's disclosure. At page 6, line 24, 
applicant specifies that "computer readable media" can include "carrier waves". These 
"carrier waves", rather than tangibly embodying the information, are merely a transient 
phenomenon in which the information signal is non-tangibly fixed. Thus, the claims 
remain directed to information perse, and are not permissible under the Interim 
Guidelines for patentable subject matter. 
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b.1. Claims 1, 4 - 6, 8 - 10, 43, 44, 52, 53 are rejected under 35 U.S.C. 102(e) as 
being anticipated by Jain et al ("Jain"; U.S. Patent Number 6,567,980). 

As per independent claim 1 (and see also independent claim 6), Jain discloses a 
VIDEO CATALOGER SYSTEM that facilitates non-linear viewing of media, the system 
comprising: 

"a scene selector that scans a digitized media and selects a scene in the 
digitized media": 

Using advanced media analysis algorithms that automatically watch, listen to 
and read a video stream, the multimedia cataloger intelligently extracts 
metadata-keyframes , time codes, textual information and an audio profile 

from the video in real-time. (col. 2 lines 10-19) 

The media analysis in Jain uses "at least one of... item recognition", to satisfy the 
alternatives list in the claim. The algorithms serve to "select" digitized media for various 
categories of inclusion. 

Jain then teaches "a metadata generator that produces metadata associated with 

the selected scene and annotates the selected scene with the metadata": 

[TJhere is a media cataloging and media analysis application which performs 
real-time, or non-real-time, indexing and distribution of video across an 
enterprise, (col. 2 lines 6-9) 

The result is that Jain provides "an organizer that places the selected annotated 

scene in a media store to facilitate non-linear viewing of one or more scenes: 

This information becomes the foundation of a rich, frame-accurate index that 
provides immediate, non-linear access to any segment of the video, (col. 2 lines 
20-23) 

As per claim 4, Jain discloses that the "recognition is adapted by a machine 
learning technique based, at least in part, on the input from the user 1 ': 
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unique metadata can be defined and added to the Video Cataloqer 1 10 by a 
user . Custom metadata tracks could include information provided in collateral 
data to the video. (Table 1 col. 7; col. 7 lines 7-12) 

These metadata additions allow the Jain user to search digital videos, and they 

are inputted by the user, after which the computer has "learned" them. 

Concerning claim 5, Jain discloses that "the metadata generator produces at 

least one of... an item name" 

The Vidsync daemons also are responsible for returning certain pieces of 
information to the Cataloger, such as the actual start time, and a digital video 
asset ID or name for later use. (col. 6 lines 2-5) 

As per claim 8, Jain discloses "selecting the scene" from the set of digitized 
media by the above-noted analysis that is applied to the video that is received for 
cataloging. This results in "automatically selecting the scene based on at least one 
of... item recognition" and, as in claim 9, in "associating at least one of... an item... with 
the selected scene" . 

Concerning claim 10 (and see also claim 36), Jain teaches at least one of, "a 

database and a datacube": 

Metadata Server 130: may be as simple as a file system containing hypertext 
markup language (HTML) files, or as complex as a relational database 
supporting a client-server application environment for media management, (col. 3 
lines 60-67) 

As per independent claim 43, Jain's metadata marking of media necessarily 
involves a "data structure that facilitates non-linear viewing of media items", the data 
structure comprising: 

"a first field that holds a media item" (to generate Figure 2's item 176); and 
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"a second field that holds a metadata item related to the media item, where the 
metadata facilitates locating a related media item by annotating the related media with 
metadata" (as in Figure 2 item 178), this for "at least one of identifying the media item". 
The metadata serves both the purposes of "identifying" (claim 52) and "locating the 
media item" (claim 53). 

Concerning independent claim 44, Jain also involves "a method of providing and 
selecting from a set of graphical user interface elements on the display" in returning 
desired content from the cataloged collection, the method comprising: 

"retrieving a set of graphical user interface elements, each of the interface 
elements representing an action associated with facilitating the non-linear display of 
media items" 

This information becomes the foundation of a rich, frame-accurate index that 
provides immediate, non-linear access to any segment of the video, (col. 2 lines 
20-24) 

"displaying the set of interface elements on the display" (Figure 2, which depicts 
a Jain GUI); 

"receiving an interface element selection signal indicative of the selection device 

selecting a selected entry from the set of interface elements" 

A panel 172 displays the live video being digitized, with play, stop, etc. controls 
that interact remotely with the analog source via a deck controller 240 (FIG. 3; 
col. 4 lines 26-28) 

"initiating processing of related media by the metadata generator and to facilitate 
non-linear viewing of media based, at least in part, upon stored metadata", this being 
the associated metadata that is used in Jain's catalog . 
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b.2. Claims 45-47 are rejected under 35 U.S.C. 102(e) as being anticipated by Morris 
("Morris"; U.S. Patent Application: 2002/0,088,000). 

As per independent claim 45 (and see also independent claim 46), Morris, in 
providing CQNTROLED ACCESS TO IMAGE METADATA , uses a "data packet 
adapted to be transmitted between two or more computer components that facilitate 
annotating a media and facilitate the non-linear viewing of the media, the data packet 
comprising:" 

"a first field that stores a clip identifier" (Figure 2 item 52), in that the header of a 
data packet is used to identify that this is a new packet, and the packet contains new 
information of the media; 

"a second field that stores a metadata key that identifies an annotating metadata 

associated with the clip identified by the clip identifier" 

The metadata for the images is preferably stored within the image file 50 in 
individual image tags 60, each of which store various types of data that 
correspond and relate to particular captured image data 54. (Figure 2 items 60 
& 62; paragraph 20); 

"a third field that stores data associated with the clip identified by the clip 

identifier" (Figure 2 item 54)— this is the field where the actual data is stored. 

As per independent claim 47, in addition to the above-noted "first field" for a "clip 

identifier" and "field that stores metadata", Morris's data arrangement also uses a "field 

that stores a requested user action concerning the portion identified by the clip 

identifier" (Figure 2 items 60 & 62; paragraph 20), since in transmitting a request for a 

metadata-annotated item in Morris, "user action" will also be specified. 
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b.3. Claims 7, 1 5 - 20, 30 - 40 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Jain in view of Duncombe et al ("Duncombe"; U.S. Patent Number 
6,813,745). 

As per claim 7, Jain discloses a method for cataloging digital media into a 
storage device to allow the user to retrieve non-linear clips. This allows the user to 
quickly search and find digital data. 

The difference between the claims and Jain is the claims recite "manually 
scanning one or more scenes from the set of digitized media; and manually selecting 
the scene." 

Duncombe teaches a MEDIA SYSTEM for storing electrical files that allow the 
user to enter search descriptions for digital retrieval similar to that of Jain. In addition, 
Duncombe discloses (col. 3 lines 13-14): FIG. 8 is a flow diagram showing the steps 
taken by a user to select and view a plurality of suitable media clips. 

It would have been obvious to one of ordinary skill in the art to modify the viewing 
GUI taught by Jain to permit the user to perform manual selections of the media file as 
per Duncombe, in order to obtain a system that automatically or manually allows the 
user to manipulate and operate a digital media storage, thus improving flexibility and 
user options available. Motivation exists at least within Jain, where the user is better 
served by having a more comprehensive search capability that accepts ongoing user 
specifications of desired data. 

Concerning independent claim 15, Jain discloses cataloging digital media into a 
storage device to allow the user to retrieve clips in a non-linear fashion. This allows the 
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user to quickly search and find digital data. In addition, Jain teaches "a scene retriever 

that retrieves one or more annotated scenes and one or more pieces of annotating 

metadata associated with the one or more annotated scenes from a media store", as 

per the metadata annotations, provided by "a metadata analyzer" that can "identify one 

or more relationships between the one or more pieces of annotating metadata": 

In one aspect of the invention, there is a media cataloging and media analysis 
application which performs real-time, or non-real-time, indexing and distribution 
of video across an enterprise. Synchronized encoding and indexing allows users 
to intelligently navigate through the video by using the index to go directly to 
the exact point of interest, rather than streaming it from start to finish (col. 2 lines 
6-9 & col. 2 lines 27-30) 

The metadata serves to find media in Jain that are of a similar nature, and thus, 
have relationships between them. 

The difference between the claims and Jain is that the claims recite "a playlist 
generator that evaluates the one or more relationships and produces a playlist of related 
scenes; one or more viewers for viewing a scene listed in the playlist; one or more 
feedback receivers for receiving a feedback concerning the viewed scene; and a playlist 
updater for updating the playlist based, at least in part on the feedback." 

Duncombe teaches a system that allows the user to browse through video clips 
by receiving user feedback, which is similar to that of Jain. In addition, Duncombe 
teaches "a playlist generator that evaluates the one or more relationships and produces 
a playlist of related scenes" (col. 2 lines 34-36): The method includes the steps of first 
organizing and formatting a plurality of media clips , "one or more viewers for viewing 
a scene listed in the playlist" (col. 2 lines 35-36): and then playing the plurality of media 
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clips based upon input supplied by the user "one or more feedback receivers for 

receiving a feedback concerning the viewed scene" to direct "a playlist updater" 

Once the plurality of suitable media clips have been played, the media system 
uses a means for soliciting feedback and a means for accepting feedback from 
the user, and a means for reselecting the plurality of suitable media clips 
based upon the feedback of the user. (col. 2 lines 50-55) 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made, to modify the viewing GUI taught by Jain to allow the user to 
search media files with the playlist and feedback control of Duncombe, in order to obtain 
a system that has the ability to generate a set of digital media clips that are in better 
accord with actual user desires. 

Concerning claims 16, 38, Jain/Duncombe discloses output on "at least one of, 
an active device and a passive device" (Duncombe, Figure 1 item 90, Figure 2 item 90, 
Jain, Figure 2). As per claim 17, the viewing arrangements of Jain/Duncombe are also 
"at least one of, an intelligent device and a non-intelligent device" (see also claim 37). 

The input in claim 18 that is "at least one of, a touch input, a typed input, a 
mouse input, a voice input and a facial expression input concerning the viewed scene" 
reads upon Duncombe (col. 6 lines 53-60): a data input mechanism 72, such as a 
keyboard 72 and a mouse 72A. A similar line of reasoning applies to claim 40. 

As per claim 19, Jain in view of Duncombe discloses that "the feedback concerns 
a current scene" and also that "the feedback comprises at least one of, a command to 
skip ahead in the playlist, a command to skip back in the playlist, a command to 
generate a new playlist, a command to find scenes similar to the current scene and a 
command to play a longer scene related to the current scene" A panel 172 displays the 
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live video being digitized, with plav. stop, etc. controls that interact remotely with the 
analog source via a deck controller 240 (Jain, col. 4 lines 27-30; FIG. 3). 

Responsive to user feedback as to desired content, the "playlist updater" of 
Duncombe "adds and/or removes a scene from the playlist based on at least one of, a 
usage data, a feedback command and a time stamp" as in claim 20 (Duncombe, Figure 
2 item 112, or Figure 13). 

As per independent claim 30, the Jain "media database" as played by the 

Duncombe "playlist" would be populated by "video segments" being "associated with an 

annotating metadata" under the control of an "annotating tool". Such Jain-style 

annotation will be "based on at least one of... item recognition", so that the annotating 

metadata comprises at least one of... an item identifier, as in claim 31. 

As per claim 32, the production of annotation data "in response to a user input" 

is at least provided by the user-operable catalog creation that is seen in Jain. As in 

claim 33, the resulting metadata is, "at least in part... an item recognition data" 

Likewise, the remaining metadata tracks (Audio Classes 324, Speech 326, 
Speaker ID 328, Keywords 330) are each a parcel of metadata spanning a time 
period, and are extracted by their corresponding feature extractor shown in FIG. 
9 (Jain, col. 6 lines 56-60). 

The input thus supplied is made part of "a machine learning technique", where 
the machine learns of user's intentions as to the media information (claim 34). 

As per independent claim 35, Jain teaches "a media data store comprising one 
or more metadata-annotated displayable items" (Figure 2 item 178), and one that 
operates in conjunction with "a presenter that presents a selected first displayable item 
from the media data store" (col. 4 lines 25-30). 
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The difference between the claim and Jain rests in the "selector that selects a 
second displayable item from the media data store based, at least in part, on a 
relationship between a first metadata associated with the first displayed item and a 
second metadata associated with the second displayable item." 

Duncombe teaches a system that allows the user to browse through video clips 
and that selects a second displayable item from the media data store based, at least in 
part, on a relationship between a first metadata associated with the first displayed item 
and a second metadata associated with the second displayable item (col. 2 lines 65-67): 
A further objective is to provide a media system that solicits feedback from the user and 
modifies the media presentation in response to the feedback. The specifics of user 
desire in Duncombe create a relationship between what is seen first and what is seen 
second. 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to modify the viewing GUI taught by Jain to include a relationship- 
establishing playback arrangement as per Duncombe, in order to obtain a system that 
will retrieve digital media segments in better accordance with user preference. This will 
then handle the situation of retrieving the second displayable item in response to the 
user's feedback concerning the "first displayable item" (claim 39). 

(10) Response to Argument 
[Ground of rejection a J 

Appellant argues at pages 6-7, concerning the "computer-readable medium", 
that "signal claims are within the provinces of statutory subject matter", and cites 
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Warmerdam. However, the possible interpretation of the "medium" as a "carrier wave" 
does not present statutory subject matter that is consistent with the current Interim 
Guidelines that have been supplied as guidance to the Examining Corps, regarding 
intangible inventions that are presented as patentable. A "carrier wave" simply does not 
represent a "machine" or "manufacture", because of its ephemeral nature. 

Appellant goes on to argue, regarding the rejection under 35 USC 101 , that 
"software code alone qualifies as an invention eligible for patenting, even if intangible 
(e.g., manifested without physical or structural components)", and cites Eolas v. 
Microsoft. However, a claim directed simply to the data content of computer instructions 
is one directed to a computer program perse, and also does not fit the categories of 
statutory invention. Current examining guidelines do not permit such claims. 
[ground of rejection b. 1.] 

Concerning Jain, appellant argues at pages 7-8 that "Jain, et a/, neither 
annotates digital media with metadata, nor stores annotated media as recited in the 
subject claims. Instead, the reference extracts metadata from the media, and stores 
that metadata separately from the media", so that "the non-linear access to various 
segments of the media requires an intermediate index". However, it remains that "a 
metadata generator" operates in Jain, in conjunction with "an organizer that places the 
selected annotated scene in a media store", as in claim 1, because the "media" in Jain, 
with its associated "metadata" as it happens to be determined, is in a state wherein it 
possesses the associated "metadata" relationships, and is thus properly interpreted as 
representing an "annotated scene". 
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[ground of rejection b.2.] 

Concerning Morris, appellant argues at page 9 that "Morris is directed toward 
providing access to metadata... but is silent on whether the user actions (as opposed to 
descriptive labels) are stored within the metadata, and also silent on whether metadata 
can be employed to adapt a clip". However, Morris provides a "data packet" with the 
"first field that stores a clip identifier" in its Header and the Image Data for the "third field 
that stores data associated with the clip". Furthermore, since the "packet" in Morris is 
used to represent user intention, the "annotating metadata" within it is found as a 
"metadata key" (claims 45, 46) and this "second field" "stores a requested user action" 
as in claim 47. Morris is a user-intensive metadata management arrangement, whose 
data structure works with packets of the kind claimed. 

At pages 9-10, appellant argues that "the data structure of Morris stores actual 
metadata , not a metadata frey", and Morris "is silent on whether metadata for one 
image can relate to other metadata". But this distinction as to the "metadata key" is not 
actually found in the claims. All that need appear in Morris is that something in a 
"packet" will point to "metadata", and this happens within a data structure like that seen 
in fig 2. This attempt to have the Examiner improperly "read in" distinctions to the term 
"key" also includes the "example" on page 10, where it "has dynamic, relational aspects 
with other data elements." 
[ground of rejection b. 3.] 

Concerning Duncombe, appellant argues at page 1 1 that "Duncombe 'organizes 
data by treating it like a four-dimensional object'..., but metadata is not contemplated by 
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the reference". However, in Duncombe, an indexing arrangement must use some form 
of "metadata" "annotation", or it will not work in finding the "media" information in a "non- 
linear" fashion. In any event, the use of "metadata" is what the Examiner relies upon 
Jain to show. Thus, even if "Duncombe selects media clips based upon relationships 
between the media as defined by a professional and/or user who have previously 
viewed the clip" (this user interaction in development of a "playlist" being one of the 
primary reasons the Examiner has relied upon Duncombe), it is not true that 
"Duncombe does not choose clips based upon relationships between one or^more 
pieces of annotating metadata ": Some form of indexing is needed to recall media in 
Duncombe, and the Examiner relies upon Jain to show actual "metadata". Appellant is 
not permitted to attack Duncombe apart from the overall combination with Jain. 

At page 12, appellant argues that "the Examiner is not combining Jain, et al. with 
a playlist generator from Duncombe, but instead with a hypothetical playlist generator 
that works in the manner the Examiner desires in order to reject the subject claims", 
since "Duncombe teaches a playlist generator, but not one that evaluates the one or 
more relationships (between metadata)". But Duncombe is directly concerned with 
finding patterns within data that is used to index media. This combines reasonably with 
Jain's metadata -based access, with at least Jain suggesting that forming relationships 
in such a way would provide a better final result to the using viewer. In Jain, "metadata" 
is the key to finding useful information that a user desires. Likewise, the generation of a 
"playlist" in Duncombe achieves this same result. Appellant argues against the 
combination of the two modes of access seen in Jain and Duncombe, when in fact, they 
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are quite compatible with one another. Jain produces the metadata, and the Duncombe 
user session helps modify its search and generates playlists. Thus, the Section 103(a) 
rejection. 

(11) Related Proceeding(s) Appendix 

No decision rendered by a court or the Board is identified by the examiner in the 
Related Appeals and Interferences section of this examiner's answer. 
Conclusion: 

For the above reasons, it is believed that the rejections under 35 USC 101 , 
102(e) and 103(a) should be sustained. 
Respectfully submitted, 



Raymond J. Bayerl 
Primary Examiner 
Art Unit 2173 
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